Applying Logging, Error Messages, and Defensive Programming Practices in Python

Course: Python Training

Topic: Applying Logging, Error Messages, and Defensive Programming Practices

Course Link: Python Training Course

Demo Link: Register for a Course Demo

Academy Website: JustAcademy

1. Introduction

Reliable Python applications need more than code that works under ideal conditions. They must detect unexpected situations, communicate errors clearly, protect data, and provide enough information to help developers investigate problems. Three important practices help achieve these goals: logging, meaningful error messages, and defensive programming.

• Logging: Records important events, warnings, errors, and diagnostic details while a program runs.
• Error messages: Explain what went wrong and, where appropriate, how the user can correct the problem.
• Defensive programming: Anticipates invalid inputs, unexpected states, and environmental failures so that the program can respond safely.

These techniques are useful in command-line tools, web applications, APIs, automation scripts, database systems, data-processing pipelines, and enterprise applications.

2. Learning Objectives

• Understand the purpose of logging and defensive programming.
• Use Python's built-in logging module.
• Choose appropriate logging levels.
• Create useful and understandable error messages.
• Apply input validation and precondition checks.
• Handle exceptions without hiding important failures.
• Protect sensitive information in logs and error messages.
• Build maintainable Python applications using practical error-handling strategies.

3. What Is Logging in Python?

Logging is the process of recording events that occur during program execution. These records help developers understand application behavior, investigate failures, monitor operations, and identify recurring problems.

Python provides a standard library module named logging. It supports different severity levels, message formatting, timestamps, and configurable output destinations such as the console or a log file.

Basic Logging Example

import logging

logging.basicConfig(level=logging.INFO)

logging.debug("Detailed diagnostic information")
logging.info("Application started")
logging.warning("Configuration value is missing")
logging.error("Unable to complete the operation")
logging.critical("A critical failure occurred")

With the logging level set to INFO, messages at the INFO level and above are normally displayed. The DEBUG message is filtered out.

4. Understanding Logging Levels

Logging levels indicate the severity or purpose of a message. Selecting the right level makes logs easier to search, filter, and interpret.

Example: Choosing Appropriate Levels

import logging

logging.basicConfig(level=logging.DEBUG)

def calculate_total(price, quantity):
    logging.debug(
        "Calculating total for price=%s, quantity=%s",
        price,
        quantity
    )

    if price < 0 or quantity < 0:
        logging.warning("Negative input received")
        raise ValueError("Price and quantity must not be negative.")

    total = price * quantity
    logging.info("Total calculated successfully")
    return total

print(calculate_total(100, 3))

In this example, DEBUG records diagnostic details, WARNING signals an unusual input, and INFO reports successful processing. The function raises an exception when its validation rule is violated.

5. Configuring the Logging System

The logging.basicConfig() function provides a simple way to configure logging for small scripts. You can set the minimum severity level, message format, output file, and file mode.

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(name)s - %(message)s",
    filename="application.log",
    encoding="utf-8"
)

logging.info("Application started")
logging.warning("A non-critical issue was detected")

This configuration sends log records to application.log. The format includes a timestamp, severity level, logger name, and message.

Important: basicConfig() generally configures logging only if the root logger does not already have handlers, unless options such as force=True are used. In larger applications, configure logging once at the application entry point rather than repeatedly inside individual functions.

6. Logging to the Console and a File

Console logging is useful during development and for services whose execution environment collects standard output or standard error. File logging is useful when logs need to be retained for later investigation.

import logging

logger = logging.getLogger("inventory")
logger.setLevel(logging.DEBUG)

console_handler = logging.StreamHandler()
console_handler.setLevel(logging.WARNING)

file_handler = logging.FileHandler(
    "inventory.log",
    encoding="utf-8"
)
file_handler.setLevel(logging.DEBUG)

formatter = logging.Formatter(
    "%(asctime)s - %(levelname)s - %(name)s - %(message)s"
)

console_handler.setFormatter(formatter)
file_handler.setFormatter(formatter)

logger.addHandler(console_handler)
logger.addHandler(file_handler)

logger.debug("Inventory calculation started")
logger.warning("Stock level is low")
logger.error("Unable to update inventory")

This example configures a named logger with two handlers. The console receives warnings and errors, while the file receives debug messages and more severe events.

For production applications, avoid adding duplicate handlers every time a module is imported or a setup function runs. Repeated configuration can cause each message to appear multiple times.

7. Writing Clear and Useful Error Messages

Error messages should explain what went wrong in a way that helps users or developers understand the problem. A useful message identifies the failed operation, describes the invalid condition, and provides a corrective action when possible.

Weak Error Message

raise ValueError("Error")

Improved Error Message

raise ValueError("Quantity must be a positive integer.")

The improved message explains the problem and the expected input. Clear error messages reduce debugging time and make applications easier to use.

Example: Validating a Payment Amount

def process_payment(amount):
    if not isinstance(amount, (int, float)) or isinstance(amount, bool):
        raise TypeError("Payment amount must be numeric.")

    if amount <= 0:
        raise ValueError("Payment amount must be greater than zero.")

    return f"Payment of {amount:.2f} accepted."

try:
    print(process_payment(250))
except (TypeError, ValueError) as error:
    print("Payment could not be processed:", error)

Best practice: User-facing messages should be understandable and actionable. Technical logs can contain additional diagnostic details, but neither logs nor user messages should expose passwords, authentication tokens, or other confidential information.

8. What Is Defensive Programming?

Defensive programming is a development approach in which code checks assumptions and handles foreseeable problems before they cause larger failures. Instead of assuming that every argument, file, API response, or database record will be valid, a defensive program verifies important conditions.

Defensive programming does not mean adding excessive checks everywhere. It means identifying realistic risks and placing appropriate safeguards at important boundaries.

Common Defensive Programming Techniques

• Validate function arguments and user input.
• Check whether files, folders, and other resources are available.
• Verify API response status and data structure.
• Handle expected exceptions explicitly.
• Use sensible defaults where appropriate.
• Protect important application invariants.
• Use type hints and documentation to communicate expectations.
• Test boundary conditions and unexpected inputs.

9. Input Validation and Preconditions

A precondition is a requirement that must be satisfied before a function performs its main operation. Input validation checks that the supplied data meets those requirements.

Example: Validating a Quantity

def calculate_order_total(price, quantity):
    if not isinstance(price, (int, float)) or isinstance(price, bool):
        raise TypeError("Price must be a number.")

    if not isinstance(quantity, int) or isinstance(quantity, bool):
        raise TypeError("Quantity must be an integer.")

    if price < 0:
        raise ValueError("Price cannot be negative.")

    if quantity <= 0:
        raise ValueError("Quantity must be greater than zero.")

    return price * quantity

try:
    total = calculate_order_total(99.50, 3)
    print("Order total:", total)
except (TypeError, ValueError) as error:
    print("Invalid order:", error)

Output:

Order total: 298.5

The function checks the type and value of each argument before calculating the total. Notice that Boolean values are explicitly excluded because bool is a subclass of int in Python.

10. Logging Exceptions with Tracebacks

When an exception occurs, logging the traceback can provide information about the sequence of function calls that led to the failure. Python's logger.exception() method is designed for use inside an exception handler.

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

logger = logging.getLogger(__name__)

def divide(a, b):
    return a / b

try:
    result = divide(10, 0)
except ZeroDivisionError:
    logger.exception("Division operation failed")
    print("Unable to complete the calculation.")

The log contains the error message and exception traceback. The user receives a simpler message without needing to see technical debugging details.

Important: Use logger.exception() inside an active exception handler. Outside a handler, use an appropriate logging method such as logger.error() for an error event that does not require a traceback.

11. Using Logger Objects in Python Applications

For maintainable applications, create a named logger rather than placing all logging calls directly on the root logger. A common convention is to use logging.getLogger(__name__) in each module.

import logging

logger = logging.getLogger(__name__)

def load_configuration(path):
    logger.info("Loading configuration")

    try:
        with open(path, "r", encoding="utf-8") as file:
            configuration = file.read()
    except FileNotFoundError:
        logger.warning("Configuration file was not found")
        raise
    except OSError:
        logger.exception("Failed to read configuration file")
        raise

    logger.info("Configuration loaded successfully")
    return configuration

This function records the start and successful completion of the operation. If the file is missing, it logs a warning and re-raises the exception. Other operating-system errors are logged with a traceback and also re-raised.

Logging and exception handling have different purposes: logging records an event, while raising or handling an exception controls the program's error flow. A log statement does not automatically handle an exception.

12. Logging with Variable Data

Log messages often need to include values such as an order ID, a filename, or a record count. The logging module supports passing variable data as separate arguments.

import logging

logger = logging.getLogger(__name__)

order_id = 1042
status = "processed"

logger.info(
    "Order %s has status %s",
    order_id,
    status
)

Passing the arguments separately allows the logging system to format the message when it is emitted. It also avoids eagerly constructing the final message when the record is filtered out.

Only log information that is useful for diagnosing or monitoring the application. Avoid logging sensitive user data or large payloads unnecessarily.

13. Defensive Programming with API Responses

External APIs may return errors, incomplete data, unexpected types, or responses that cannot be decoded. Defensive code validates both the response status and the data it expects to use.

def get_user_name(response_data):
    if not isinstance(response_data, dict):
        raise TypeError("API response must be a dictionary.")

    user = response_data.get("user")

    if not isinstance(user, dict):
        raise ValueError("API response does not contain valid user data.")

    name = user.get("name")

    if not isinstance(name, str) or not name.strip():
        raise ValueError("User name is missing or invalid.")

    return name.strip()

sample_response = {
    "user": {
        "name": "Aarav"
    }
}

try:
    print(get_user_name(sample_response))
except (TypeError, ValueError) as error:
    print("Unable to process API response:", error)

Output:

Aarav

This example validates the structure of the data before accessing nested values. In a real API client, also check the HTTP status, configure timeouts, handle network exceptions, and validate the response according to the API's documented schema.

14. Defensive Programming with File Operations

File operations can fail because a file does not exist, the application lacks permission, the storage device is unavailable, or the path is invalid. A defensive program handles relevant failures instead of assuming that every operation will succeed.

import logging

logger = logging.getLogger(__name__)

def read_text_file(path):
    try:
        with open(path, "r", encoding="utf-8") as file:
            content = file.read()
    except FileNotFoundError:
        logger.warning("Requested file was not found")
        raise
    except PermissionError:
        logger.exception("Permission denied while reading file")
        raise
    except OSError:
        logger.exception("File system error occurred")
        raise
    else:
        logger.info("File read successfully")
        return content

try:
    text = read_text_file("notes.txt")
    print(text)
except OSError:
    print("Unable to read the requested file.")

The with statement manages file closure automatically. The except blocks record or propagate failures, while else logs success only after the file has been read without an exception.

15. Avoiding Overly Broad Exception Handling

Catching every exception with except Exception can hide programming mistakes if the code simply ignores the error. Prefer catching specific exceptions when you know which failures can occur and how to handle them.

Problematic Example

try:
    result = 10 / 0
except Exception:
    pass

print("The operation completed.")

This code suppresses the division error and continues, even though the operation did not complete successfully. The final message is misleading.

Improved Example

try:
    result = 10 / 0
except ZeroDivisionError as error:
    print("Calculation failed:", error)
else:
    print("Result:", result)

The improved version handles the specific exception and does not claim success after a failed calculation. If the application cannot recover, it may be more appropriate to log the failure and re-raise it.

16. Fail Fast and Maintain Program Invariants

Fail fast means detecting invalid states as early as practical and reporting them clearly. An invariant is a condition that should remain true throughout an object's or application's operation.

class BankAccount:
    def __init__(self, opening_balance=0):
        if opening_balance < 0:
            raise ValueError("Opening balance cannot be negative.")

        self.balance = opening_balance

    def withdraw(self, amount):
        if amount <= 0:
            raise ValueError("Withdrawal must be positive.")

        if amount > self.balance:
            raise ValueError("Insufficient balance.")

        self.balance -= amount
        return self.balance

account = BankAccount(1000)
print(account.withdraw(200))

Output:

800

The class prevents invalid opening balances and withdrawals. This makes the object's rules explicit and reduces the chance of invalid state changes.

17. Logging, Error Messages, and Defensive Programming Compared

These practices work best together. Defensive checks detect a problem, exceptions communicate failure through the program, clear messages explain the issue, and logging preserves diagnostic details where appropriate.

18. Security and Privacy in Logging

Logs can contain operational information that is useful to developers, but they can also expose confidential data if designed carelessly.

• Never log passwords, secret keys, access tokens, or authentication credentials.
• Avoid recording full payment-card information or unnecessary personal data.
• Sanitize untrusted text when it could introduce misleading log entries or control characters.
• Restrict access to log files and log-management systems.
• Use suitable retention policies so logs are not stored indefinitely without a reason.
• Prefer stable identifiers over unnecessary personal details.
• Separate detailed internal diagnostics from messages shown to end users.
• Ensure log failures do not silently cause critical business operations to be treated as successful.

19. Common Mistakes and How to Fix Them

20. Best Practices Checklist

• Use named loggers such as logging.getLogger(__name__).
• Choose a suitable level for each event.
• Keep messages concise, informative, and consistent.
• Use logger.exception() inside exception handlers when a traceback is useful.
• Validate inputs at application boundaries.
• Raise appropriate exceptions for invalid arguments or failed operations.
• Handle exceptions only where meaningful recovery or translation is possible.
• Use with for files and other resources that support context managers.
• Do not reveal secrets in error messages or logs.
• Test successful cases, failure cases, and boundary conditions.
• Configure logging in the application's entry point rather than repeatedly in every function.
• Document important exceptions and validation requirements.

21. Practice Exercises

1. Create a calculator that logs successful calculations and raises an exception when the divisor is zero.
2. Write a function that validates a user's age and raises a meaningful ValueError for invalid values.
3. Read a file using a context manager, handle missing-file errors, and log the outcome.
4. Build a login-validation function that records failed attempts without logging passwords.
5. Validate an API-like dictionary and raise an exception when a required field is missing.
6. Configure logging to write timestamps, logger names, severity levels, and messages to a file.
7. Write a function that catches an expected exception, logs the traceback, and re-raises the exception.

22. Mini Project: Defensive Order Processing System

Project Objective: Develop a small order-processing function that validates inputs, logs important events, raises meaningful exceptions, and reports failures safely.

Requirements

• Accept a product name, unit price, and quantity.
• Validate the product name and numeric inputs.
• Reject negative prices and non-positive quantities.
• Calculate the total only after validation succeeds.
• Log successful order calculations.
• Log unexpected calculation failures with diagnostic information.
• Show a clear error message without exposing confidential data.

Sample Solution

import logging

logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s - %(levelname)s - %(message)s"
)

logger = logging.getLogger(__name__)

def calculate_order(product_name, price, quantity):
    if not isinstance(product_name, str) or not product_name.strip():
        raise ValueError("Product name must not be empty.")

    if isinstance(price, bool) or not isinstance(price, (int, float)):
        raise TypeError("Price must be numeric.")

    if isinstance(quantity, bool) or not isinstance(quantity, int):
        raise TypeError("Quantity must be an integer.")

    if price < 0:
        raise ValueError("Price cannot be negative.")

    if quantity <= 0:
        raise ValueError("Quantity must be greater than zero.")

    total = price * quantity

    logger.info(
        "Order total calculated for product=%s, quantity=%s",
        product_name.strip(),
        quantity
    )

    return {
        "product": product_name.strip(),
        "quantity": quantity,
        "total": total
    }

try:
    order = calculate_order("Notebook", 50, 4)
except (TypeError, ValueError) as error:
    logger.warning("Order validation failed: %s", error)
    print("Unable to process the order:", error)
except Exception:
    logger.exception("Unexpected order-processing failure")
    print("An unexpected error occurred.")
else:
    print("Order processed successfully:", order)

Sample Output:

Order processed successfully: {'product': 'Notebook', 'quantity': 4, 'total': 200}

The example separates validation, logging, exception handling, and success reporting. In a production application, log identifiers should be chosen carefully, and unexpected exceptions should generally be handled at an appropriate application boundary rather than swallowed deep inside business logic.

23. Interview Questions and Answers

Q1. What is logging in Python?

Logging records application events at different severity levels to help developers monitor behavior and investigate problems.

Q2. What is the difference between logging and print?

print() displays text, while the logging module provides severity levels, configurable destinations, formatting, and diagnostic features.

Q3. What are the standard logging levels?

The main levels, from lowest to highest severity, are DEBUG, INFO, WARNING, ERROR, and CRITICAL.

Q4. What is defensive programming?

Defensive programming anticipates realistic errors and invalid conditions through validation, precondition checks, appropriate exception handling, and safe resource management.

Q5. Why should exceptions have meaningful messages?

Clear messages explain the cause of failure and help developers or users determine what needs to be corrected.

Q6. What does logger.exception() do?

It logs a message at error level and includes exception information, typically a traceback. It is intended for use inside an exception handler.

Q7. Why should broad exception handling be avoided?

Broad handlers can conceal unexpected programming errors or make failed operations appear successful. Catch specific exceptions whenever practical.

Q8. What is the difference between logging an exception and raising it?

Logging records diagnostic information. Raising an exception changes program control flow and signals that the operation failed. Logging alone does not handle or resolve the failure.

Q9. How can logs be made safer?

Exclude secrets, minimize personal information, sanitize untrusted data, restrict log access, and apply appropriate retention policies.

Q10. Why are automated tests important for defensive programming?

Tests verify that valid inputs work correctly and that invalid inputs, boundary conditions, and expected failures are handled consistently.

24. Quick Revision

• Logging: Records events for troubleshooting and monitoring.
• Error messages: Explain failures clearly and safely.
• Defensive programming: Checks assumptions before they cause larger problems.
• Logger levels: Use DEBUG, INFO, WARNING, ERROR, and CRITICAL appropriately.
• Exception handling: Catch expected errors where the program can respond meaningfully.
• Validation: Check types, ranges, required fields, and business rules.
• Security: Never expose secrets in diagnostic messages.
• Testing: Verify both successful execution and failure scenarios.

25. Conclusion

Logging, clear error messages, and defensive programming are essential for building reliable and maintainable Python applications. Logging provides diagnostic visibility, error messages explain failures, and defensive checks prevent invalid data or unexpected conditions from causing avoidable problems. Applying these practices together makes software easier to troubleshoot, test, secure, and maintain.

Continue learning through the JustAcademy Python Training Course, register for a session through the course demo registration page, or explore the JustAcademy website.

Further Reading: Official Python Logging HOWTO.